2026-04-30 15:55:46,246 [ 48570 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:42, check_args_and_update_paths) 2026-04-30 15:55:46,246 [ 48570 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:86, check_args_and_update_paths) 2026-04-30 15:55:46,247 [ 48570 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:97, check_args_and_update_paths) 2026-04-30 15:55:46,247 [ 48570 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:99, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_i8td71 --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=1e0b53d756cf -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_accept_invalid_certificate/test.py::test_accept test_accept_invalid_certificate/test.py::test_connection_accept test_accept_invalid_certificate/test.py::test_default test_accept_invalid_certificate/test.py::test_strict_connection_reject test_accept_invalid_certificate/test.py::test_strict_reject test_accept_invalid_certificate/test.py::test_strict_reject_with_config test_access_control_on_cluster/test.py::test_access_control_on_cluster test_access_control_on_cluster/test.py::test_grant_all_on_cluster test_access_control_on_cluster/test.py::test_grant_current_database_on_cluster test_async_insert_memory/test.py::test_memory_usage test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field 'test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Atomic]' 'test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Ordinary]' test_backup_log/test.py::test_backup_log 'test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3]' 'test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3_other_bucket]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_local]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3_plain]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_local]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_s3_plain]' test_backup_restore_s3/test.py::test_backup_to_s3 test_backup_restore_s3/test.py::test_backup_to_s3_different_credentials test_backup_restore_s3/test.py::test_backup_to_s3_multipart test_backup_restore_s3/test.py::test_backup_to_s3_named_collection 'test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3]' 'test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_other_bucket]' 'test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_plain_rewritable]' test_backup_restore_s3/test.py::test_backup_to_s3_native_copy_multipart test_backup_restore_s3/test.py::test_backup_to_tar test_backup_restore_s3/test.py::test_backup_to_tar_bz2 test_backup_restore_s3/test.py::test_backup_to_tar_gz test_backup_restore_s3/test.py::test_backup_to_tar_lzma test_backup_restore_s3/test.py::test_backup_to_tar_xz test_backup_restore_s3/test.py::test_backup_to_tar_zst test_backup_restore_s3/test.py::test_backup_to_zip 'test_backup_restore_s3/test.py::test_backup_with_fs_cache[False-True-False]' 'test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-False-False]' 'test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-False]' 'test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-True]' test_backup_restore_s3/test.py::test_incremental_backup_append_table_def test_backup_restore_s3/test.py::test_user_specific_auth test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability test_compression_nested_columns/test.py::test_nested_compression_codec test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool test_config_reloader_interval/test.py::test_reload_config test_consistant_parts_after_move_partition/test.py::test_consistent_part_after_move_partition test_dictionaries_access/test.py::test_create test_dictionaries_access/test.py::test_dictget test_dictionaries_access/test.py::test_drop 'test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_complex[complex_key_cache]' 'test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_simple[cache]' test_dictionaries_config_reload/test.py::test 'test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary0-instances_to_check0]' 'test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary1-instances_to_check1]' 'test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary2-instances_to_check2]' 'test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary3-instances_to_check3]' test_dictionaries_with_invalid_structure/test.py::test_dictGet_func_for_invalid_dictionary test_dictionaries_with_invalid_structure/test.py::test_select_from_system_dictionaries_with_invalid_dictionary 'test_drop_is_lock_free/test.py::test_query_is_lock_free[detach part]' 'test_drop_is_lock_free/test.py::test_query_is_lock_free[detach table]' 'test_drop_is_lock_free/test.py::test_query_is_lock_free[drop part]' 'test_drop_is_lock_free/test.py::test_query_is_lock_free[truncate]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-NoTx]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxCommit]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxNotFinished]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxRollback]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-NoTx]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxCommit]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxNotFinished]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxRollback]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-NoTx]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxCommit]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxNotFinished]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxRollback]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-NoTx]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxCommit]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxNotFinished]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxRollback]' test_executable_user_defined_function/test.py::test_executable_function_always_error_python test_executable_user_defined_function/test.py::test_executable_function_argument_python test_executable_user_defined_function/test.py::test_executable_function_bash test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash test_executable_user_defined_function/test.py::test_executable_function_parameter_python test_executable_user_defined_function/test.py::test_executable_function_python test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python test_executable_user_defined_function/test.py::test_executable_function_signalled_python test_executable_user_defined_function/test.py::test_executable_function_slow_python test_executable_user_defined_function/test.py::test_executable_function_sum_json_python test_executable_user_defined_function/test.py::test_executable_function_sum_python test_extreme_deduplication/test.py::test_deduplication_window_in_seconds test_failed_async_inserts/test.py::test_failed_async_inserts 'test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PART-2020-08-28-20200828_0_0_0]' 'test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PARTITION-2020-08-27-2020-08-27]' test_groupBitmapAnd_on_distributed/test.py::test_groupBitmapAnd_on_distributed_table -vvv" altinityinfra/integration-tests-runner:37a9815fd2fa '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: order-1.0.1, random-0.2, timeout-2.2.0, repeat-0.9.3, reportlog-0.4.0, xdist-3.5.0 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Atomic] test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_complex[complex_key_cache] test_executable_user_defined_function/test.py::test_executable_function_always_error_python test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3] test_access_control_on_cluster/test.py::test_access_control_on_cluster test_drop_is_lock_free/test.py::test_query_is_lock_free[detach part] test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary0-instances_to_check0] test_accept_invalid_certificate/test.py::test_accept test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_dictionaries_access/test.py::test_create [gw5] [ 1%] PASSED test_accept_invalid_certificate/test.py::test_accept test_accept_invalid_certificate/test.py::test_connection_accept [gw5] [ 2%] PASSED test_accept_invalid_certificate/test.py::test_connection_accept test_accept_invalid_certificate/test.py::test_default [gw5] [ 3%] PASSED test_accept_invalid_certificate/test.py::test_default test_accept_invalid_certificate/test.py::test_strict_connection_reject [gw5] [ 4%] PASSED test_accept_invalid_certificate/test.py::test_strict_connection_reject test_accept_invalid_certificate/test.py::test_strict_reject [gw5] [ 5%] PASSED test_accept_invalid_certificate/test.py::test_strict_reject test_accept_invalid_certificate/test.py::test_strict_reject_with_config [gw5] [ 6%] PASSED test_accept_invalid_certificate/test.py::test_strict_reject_with_config [gw0] [ 7%] PASSED test_drop_is_lock_free/test.py::test_query_is_lock_free[detach part] test_drop_is_lock_free/test.py::test_query_is_lock_free[detach table] [gw4] [ 8%] PASSED test_dictionaries_access/test.py::test_create test_async_insert_memory/test.py::test_memory_usage test_dictionaries_access/test.py::test_dictget [gw4] [ 9%] PASSED test_dictionaries_access/test.py::test_dictget [gw8] [ 10%] PASSED test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary0-instances_to_check0] [gw7] [ 11%] PASSED test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Atomic] test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Ordinary] test_dictionaries_access/test.py::test_drop test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary1-instances_to_check1] [gw3] [ 12%] PASSED test_executable_user_defined_function/test.py::test_executable_function_always_error_python test_executable_user_defined_function/test.py::test_executable_function_argument_python [gw0] [ 13%] PASSED test_drop_is_lock_free/test.py::test_query_is_lock_free[detach table] test_drop_is_lock_free/test.py::test_query_is_lock_free[drop part] [gw4] [ 14%] PASSED test_dictionaries_access/test.py::test_drop [gw3] [ 15%] PASSED test_executable_user_defined_function/test.py::test_executable_function_argument_python test_executable_user_defined_function/test.py::test_executable_function_bash [gw8] [ 16%] PASSED test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary1-instances_to_check1] test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary2-instances_to_check2] [gw3] [ 17%] PASSED test_executable_user_defined_function/test.py::test_executable_function_bash test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python [gw0] [ 18%] PASSED test_drop_is_lock_free/test.py::test_query_is_lock_free[drop part] test_drop_is_lock_free/test.py::test_query_is_lock_free[truncate] [gw3] [ 19%] PASSED test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash [gw7] [ 20%] PASSED test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Ordinary] test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic [gw3] [ 21%] PASSED test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash test_executable_user_defined_function/test.py::test_executable_function_parameter_python [gw6] [ 22%] PASSED test_access_control_on_cluster/test.py::test_access_control_on_cluster test_access_control_on_cluster/test.py::test_grant_all_on_cluster [gw8] [ 23%] PASSED test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary2-instances_to_check2] test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary3-instances_to_check3] [gw3] [ 24%] PASSED test_executable_user_defined_function/test.py::test_executable_function_parameter_python test_executable_user_defined_function/test.py::test_executable_function_python [gw6] [ 25%] PASSED test_access_control_on_cluster/test.py::test_grant_all_on_cluster test_access_control_on_cluster/test.py::test_grant_current_database_on_cluster [gw0] [ 26%] PASSED test_drop_is_lock_free/test.py::test_query_is_lock_free[truncate] test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-NoTx] [gw3] [ 27%] PASSED test_executable_user_defined_function/test.py::test_executable_function_python test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python test_dictionaries_with_invalid_structure/test.py::test_dictGet_func_for_invalid_dictionary [gw3] [ 28%] PASSED test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python test_executable_user_defined_function/test.py::test_executable_function_signalled_python [gw6] [ 29%] PASSED test_access_control_on_cluster/test.py::test_grant_current_database_on_cluster [gw8] [ 30%] PASSED test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary3-instances_to_check3] [gw3] [ 31%] PASSED test_executable_user_defined_function/test.py::test_executable_function_signalled_python test_executable_user_defined_function/test.py::test_executable_function_slow_python [gw3] [ 32%] PASSED test_executable_user_defined_function/test.py::test_executable_function_slow_python test_executable_user_defined_function/test.py::test_executable_function_sum_json_python test_consistant_parts_after_move_partition/test.py::test_consistent_part_after_move_partition test_compression_nested_columns/test.py::test_nested_compression_codec [gw3] [ 33%] PASSED test_executable_user_defined_function/test.py::test_executable_function_sum_json_python test_executable_user_defined_function/test.py::test_executable_function_sum_python [gw1] [ 34%] PASSED test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3] test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3_other_bucket] [gw3] [ 35%] PASSED test_executable_user_defined_function/test.py::test_executable_function_sum_python [gw0] [ 36%] PASSED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-NoTx] test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxCommit] [gw7] [ 37%] PASSED test_dictionaries_with_invalid_structure/test.py::test_dictGet_func_for_invalid_dictionary test_dictionaries_with_invalid_structure/test.py::test_select_from_system_dictionaries_with_invalid_dictionary [gw7] [ 38%] PASSED test_dictionaries_with_invalid_structure/test.py::test_select_from_system_dictionaries_with_invalid_dictionary test_groupBitmapAnd_on_distributed/test.py::test_groupBitmapAnd_on_distributed_table [gw1] [ 39%] PASSED test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3_other_bucket] test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_local] test_failed_async_inserts/test.py::test_failed_async_inserts [gw1] [ 40%] PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_local] test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3] [gw9] [ 41%] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_complex[complex_key_cache] test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_simple[cache] [gw1] [ 42%] PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3] test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3_plain] [gw8] [ 43%] PASSED test_compression_nested_columns/test.py::test_nested_compression_codec [gw0] [ 44%] PASSED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxCommit] test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxNotFinished] [gw1] [ 45%] PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3_plain] test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_local] [gw2] [ 46%] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_dictionaries_config_reload/test.py::test [gw3] [ 47%] PASSED test_groupBitmapAnd_on_distributed/test.py::test_groupBitmapAnd_on_distributed_table [gw1] [ 48%] PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_local] test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_s3_plain] [gw7] [ 49%] PASSED test_failed_async_inserts/test.py::test_failed_async_inserts [gw1] [ 50%] PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_s3_plain] test_backup_restore_s3/test.py::test_backup_to_s3 [gw1] [ 51%] PASSED test_backup_restore_s3/test.py::test_backup_to_s3 test_backup_restore_s3/test.py::test_backup_to_s3_different_credentials [gw8] [ 52%] PASSED test_dictionaries_config_reload/test.py::test [gw1] [ 53%] PASSED test_backup_restore_s3/test.py::test_backup_to_s3_different_credentials test_backup_restore_s3/test.py::test_backup_to_s3_multipart [gw6] [ 54%] PASSED test_consistant_parts_after_move_partition/test.py::test_consistent_part_after_move_partition [gw9] [ 55%] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_simple[cache] test_extreme_deduplication/test.py::test_deduplication_window_in_seconds test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PART-2020-08-28-20200828_0_0_0] [gw1] [ 56%] PASSED test_backup_restore_s3/test.py::test_backup_to_s3_multipart test_backup_restore_s3/test.py::test_backup_to_s3_named_collection [gw1] [ 57%] PASSED test_backup_restore_s3/test.py::test_backup_to_s3_named_collection test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3] [gw1] [ 58%] PASSED test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3] test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_other_bucket] [gw1] [ 59%] PASSED test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_other_bucket] test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_plain_rewritable] [gw9] [ 60%] PASSED test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PART-2020-08-28-20200828_0_0_0] test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PARTITION-2020-08-27-2020-08-27] [gw9] [ 61%] PASSED test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PARTITION-2020-08-27-2020-08-27] [gw6] [ 62%] PASSED test_extreme_deduplication/test.py::test_deduplication_window_in_seconds [gw1] [ 63%] PASSED test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_plain_rewritable] test_backup_restore_s3/test.py::test_backup_to_s3_native_copy_multipart [gw0] [ 64%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxNotFinished] test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxRollback] [gw0] [ 65%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxRollback] test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-NoTx] [gw0] [ 66%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-NoTx] test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxCommit] [gw0] [ 67%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxCommit] test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxNotFinished] [gw0] [ 68%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxNotFinished] test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxRollback] [gw0] [ 69%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxRollback] test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-NoTx] [gw0] [ 70%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-NoTx] test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxCommit] [gw0] [ 71%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxCommit] test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxNotFinished] [gw0] [ 72%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxNotFinished] test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxRollback] [gw0] [ 73%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxRollback] test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-NoTx] [gw0] [ 74%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-NoTx] test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxCommit] [gw0] [ 75%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxCommit] test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxNotFinished] [gw0] [ 76%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxNotFinished] test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxRollback] [gw0] [ 77%] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxRollback] [gw2] [ 78%] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas [gw1] [ 79%] PASSED test_backup_restore_s3/test.py::test_backup_to_s3_native_copy_multipart test_backup_restore_s3/test.py::test_backup_to_tar [gw1] [ 80%] PASSED test_backup_restore_s3/test.py::test_backup_to_tar test_backup_restore_s3/test.py::test_backup_to_tar_bz2 [gw1] [ 81%] PASSED test_backup_restore_s3/test.py::test_backup_to_tar_bz2 test_backup_restore_s3/test.py::test_backup_to_tar_gz [gw1] [ 82%] PASSED test_backup_restore_s3/test.py::test_backup_to_tar_gz test_backup_restore_s3/test.py::test_backup_to_tar_lzma [gw1] [ 83%] PASSED test_backup_restore_s3/test.py::test_backup_to_tar_lzma test_backup_restore_s3/test.py::test_backup_to_tar_xz [gw1] [ 84%] PASSED test_backup_restore_s3/test.py::test_backup_to_tar_xz test_backup_restore_s3/test.py::test_backup_to_tar_zst [gw1] [ 85%] PASSED test_backup_restore_s3/test.py::test_backup_to_tar_zst test_backup_restore_s3/test.py::test_backup_to_zip [gw4] [ 86%] PASSED test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic [gw1] [ 87%] PASSED test_backup_restore_s3/test.py::test_backup_to_zip test_backup_restore_s3/test.py::test_backup_with_fs_cache[False-True-False] test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability [gw1] [ 88%] PASSED test_backup_restore_s3/test.py::test_backup_with_fs_cache[False-True-False] test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-False-False] [gw5] [ 89%] FAILED test_async_insert_memory/test.py::test_memory_usage [gw1] [ 90%] PASSED test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-False-False] test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-False] test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field [gw1] [ 91%] PASSED test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-False] test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-True] [gw1] [ 92%] PASSED test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-True] test_backup_restore_s3/test.py::test_incremental_backup_append_table_def [gw4] [ 93%] PASSED test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability [gw1] [ 94%] PASSED test_backup_restore_s3/test.py::test_incremental_backup_append_table_def test_backup_restore_s3/test.py::test_user_specific_auth test_config_reloader_interval/test.py::test_reload_config [gw2] [ 95%] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool [gw5] [ 96%] PASSED test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field [gw1] [ 97%] PASSED test_backup_restore_s3/test.py::test_user_specific_auth test_backup_log/test.py::test_backup_log [gw2] [ 98%] PASSED test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool [gw4] [ 99%] PASSED test_config_reloader_interval/test.py::test_reload_config [gw5] [100%] PASSED test_backup_log/test.py::test_backup_log =================================== FAILURES =================================== ______________ test_query_is_permanent[detach-part-TxNotFinished] ______________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxNotFinished' permanent = ("ALTER TABLE {table} DETACH PARTITION '20221001';", 49) exclusive_table = 'table_test_query_is_permanent_detach_part_TxNotFinished_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) node.query( f""" INSERT INTO {exclusive_table} SELECT number, toDate('2022-09-01') + INTERVAL number DAY FROM numbers(50); """ ) query_id = "select-" + exclusive_table select_handler = node.get_query_request( f""" SELECT sleepEachRow(3) FROM {exclusive_table} SETTINGS function_sleep_max_microseconds_per_block = 0, max_threads=1; """, query_id=query_id, ) wait_select_start(query_id) for _ in [1, 2, 3, 4, 5]: assert is_query_running(query_id) assert select_handler.process.poll() is None time.sleep(1) permanent_query = permanent[0] result = permanent[1] statement = permanent_query.format(table=exclusive_table) if transaction == "TxCommit": query = f""" BEGIN TRANSACTION; {statement} COMMIT; """ elif transaction == "TxRollback": query = f""" BEGIN TRANSACTION; {statement} ROLLBACK; """ result = 50 elif transaction == "TxNotFinished": query = f""" BEGIN TRANSACTION; {statement} """ result = 50 else: query = statement node.query(query) > node.restart_clickhouse(kill=True) test_drop_is_lock_free/test.py:217: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:4055: in restart_clickhouse self.start_clickhouse(stop_start_wait_sec) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = start_wait_sec = 60, retry_start = True, expected_to_fail = False def start_clickhouse( self, start_wait_sec=60, retry_start=True, expected_to_fail=False ): if not self.stay_alive: raise Exception( "ClickHouse can be started again only with stay_alive=True instance" ) start_time = time.time() time_to_sleep = 0.5 while start_time + start_wait_sec >= time.time(): # sometimes after SIGKILL (hard reset) server may refuse to start for some time # for different reasons. pid = self.get_process_pid("clickhouse") if pid is None: logging.debug("No clickhouse process running. Start new one.") self.exec_in_container( ["bash", "-c", "{} --daemon".format(self.clickhouse_start_command)], user=str(os.getuid()), ) if expected_to_fail: self.wait_start_failed(start_wait_sec + start_time - time.time()) return time.sleep(1) continue else: logging.debug("Clickhouse process running.") if expected_to_fail: raise Exception("ClickHouse was expected not to be running.") try: self.wait_start(start_wait_sec + start_time - time.time()) return except Exception as e: logging.warning( f"Current start attempt failed. Will kill {pid} just in case." ) self.exec_in_container( ["bash", "-c", f"kill -9 {pid}"], user="root", nothrow=True ) if not retry_start: raise time.sleep(time_to_sleep) > raise Exception("Cannot start ClickHouse, see additional info in logs") E Exception: Cannot start ClickHouse, see additional info in logs helpers/cluster.py:3992: Exception ------------------------------ Captured log call ------------------------------- 2026-04-30 16:02:03 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_detach_part_TxNotFinished_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) 2026-04-30 16:02:05 [ 389 ] DEBUG : Executing query INSERT INTO table_test_query_is_permanent_detach_part_TxNotFinished_ SELECT number, toDate('2022-09-01') + INTERVAL number DAY FROM numbers(50); on node (cluster.py:3602, query) 2026-04-30 16:02:09 [ 389 ] DEBUG : Executing query SELECT sleepEachRow(3) FROM table_test_query_is_permanent_detach_part_TxNotFinished_ SETTINGS function_sleep_max_microseconds_per_block = 0, max_threads=1; on node (cluster.py:3667, get_query_request) 2026-04-30 16:02:10 [ 389 ] DEBUG : Executing query SELECT count() FROM system.processes WHERE query_id = 'select-table_test_query_is_permanent_detach_part_TxNotFinished_'; on node (cluster.py:3602, query) 2026-04-30 16:02:12 [ 389 ] DEBUG : Executing query SELECT count() FROM system.processes WHERE query_id = 'select-table_test_query_is_permanent_detach_part_TxNotFinished_'; on node (cluster.py:3602, query) 2026-04-30 16:02:16 [ 389 ] DEBUG : Executing query SELECT count() FROM system.processes WHERE query_id = 'select-table_test_query_is_permanent_detach_part_TxNotFinished_'; on node (cluster.py:3602, query) 2026-04-30 16:02:20 [ 389 ] DEBUG : Executing query SELECT count() FROM system.processes WHERE query_id = 'select-table_test_query_is_permanent_detach_part_TxNotFinished_'; on node (cluster.py:3602, query) 2026-04-30 16:02:24 [ 389 ] DEBUG : Executing query SELECT count() FROM system.processes WHERE query_id = 'select-table_test_query_is_permanent_detach_part_TxNotFinished_'; on node (cluster.py:3602, query) 2026-04-30 16:02:27 [ 389 ] DEBUG : Executing query SELECT count() FROM system.processes WHERE query_id = 'select-table_test_query_is_permanent_detach_part_TxNotFinished_'; on node (cluster.py:3602, query) 2026-04-30 16:02:29 [ 389 ] DEBUG : Executing query BEGIN TRANSACTION; ALTER TABLE table_test_query_is_permanent_detach_part_TxNotFinished_ DETACH PARTITION '20221001'; on node (cluster.py:3602, query) 2026-04-30 16:02:31 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] (cluster.py:2173, exec_in_container) 2026-04-30 16:02:31 [ 389 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', 'ps -C clickhouse'] (cluster.py:113, run_and_check) 2026-04-30 16:02:32 [ 389 ] DEBUG : Stdout: PID TTY TIME CMD (cluster.py:121, run_and_check) 2026-04-30 16:02:32 [ 389 ] DEBUG : Stdout: 1625 ? 00:01:33 clickhouse (cluster.py:121, run_and_check) 2026-04-30 16:02:32 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] (cluster.py:2173, exec_in_container) 2026-04-30 16:02:32 [ 389 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', 'pkill -9 clickhouse'] (cluster.py:113, run_and_check) 2026-04-30 16:02:33 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:02:33 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:02:34 [ 389 ] DEBUG : Stdout:1625 (cluster.py:121, run_and_check) 2026-04-30 16:02:35 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:02:35 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:02:36 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:02:36 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:02:38 [ 389 ] DEBUG : No clickhouse process running. Start new one. (cluster.py:3964, start_clickhouse) 2026-04-30 16:02:38 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon'] (cluster.py:2173, exec_in_container) 2026-04-30 16:02:38 [ 389 ] DEBUG : Command:['docker', 'exec', '-u', '0', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon'] (cluster.py:113, run_and_check) 2026-04-30 16:02:40 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:02:40 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:02:41 [ 389 ] DEBUG : Stdout:2511 (cluster.py:121, run_and_check) 2026-04-30 16:02:41 [ 389 ] DEBUG : Clickhouse process running. (cluster.py:3975, start_clickhouse) 2026-04-30 16:02:41 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:02:41 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:02:43 [ 389 ] DEBUG : Stdout:2511 (cluster.py:121, run_and_check) 2026-04-30 16:02:43 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:02:46 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:02:47 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:02:49 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:02:50 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:02:52 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:02:54 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:02:55 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:02:57 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:02:58 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:01 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:03:01 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:03:02 [ 389 ] DEBUG : Stdout:2511 (cluster.py:121, run_and_check) 2026-04-30 16:03:02 [ 389 ] WARNING : ERROR Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) (cluster.py:4008, wait_start) 2026-04-30 16:03:02 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:03:02 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:03:03 [ 389 ] DEBUG : Stdout:2511 (cluster.py:121, run_and_check) 2026-04-30 16:03:03 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:05 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:07 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:08 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:10 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:11 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:12 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:15 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:16 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:18 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:20 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:03:20 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:03:21 [ 389 ] DEBUG : Stdout:2511 (cluster.py:121, run_and_check) 2026-04-30 16:03:21 [ 389 ] WARNING : ERROR Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) (cluster.py:4008, wait_start) 2026-04-30 16:03:21 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:03:21 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:03:22 [ 389 ] DEBUG : Stdout:2511 (cluster.py:121, run_and_check) 2026-04-30 16:03:22 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:24 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:26 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:28 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:30 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:31 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:35 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:37 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:39 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:42 [ 389 ] DEBUG : Executing query select 20 on node (cluster.py:3602, query) 2026-04-30 16:03:43 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:03:43 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:03:44 [ 389 ] DEBUG : Stdout:2511 (cluster.py:121, run_and_check) 2026-04-30 16:03:44 [ 389 ] WARNING : ERROR Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) (cluster.py:4008, wait_start) 2026-04-30 16:03:44 [ 389 ] ERROR : No time left to start. But process is still running. Will dump threads. (cluster.py:4013, wait_start) 2026-04-30 16:03:44 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] (cluster.py:2173, exec_in_container) 2026-04-30 16:03:44 [ 389 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', 'ps -C clickhouse'] (cluster.py:113, run_and_check) 2026-04-30 16:03:46 [ 389 ] DEBUG : Stdout: PID TTY TIME CMD (cluster.py:121, run_and_check) 2026-04-30 16:03:46 [ 389 ] DEBUG : Stdout: 2511 ? 00:01:23 clickhouse (cluster.py:121, run_and_check) 2026-04-30 16:03:46 [ 389 ] INFO : PS RESULT: PID TTY TIME CMD 2511 ? 00:01:23 clickhouse (cluster.py:4019, wait_start) 2026-04-30 16:03:46 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-04-30 16:03:46 [ 389 ] DEBUG : Command:['docker', 'exec', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-04-30 16:03:48 [ 389 ] DEBUG : Stdout:2511 (cluster.py:121, run_and_check) 2026-04-30 16:03:48 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:False cmd: ['bash', '-c', "gdb -batch -ex 'thread apply all bt full' -p 2511"] (cluster.py:2173, exec_in_container) 2026-04-30 16:03:48 [ 389 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', "gdb -batch -ex 'thread apply all bt full' -p 2511"] (cluster.py:113, run_and_check) 2026-04-30 16:08:48 [ 389 ] WARNING : Current start attempt failed. Will kill 2511 just in case. (cluster.py:3982, start_clickhouse) 2026-04-30 16:08:48 [ 389 ] DEBUG : run container_id:roottestdropislockfree_gw0_node_1 detach:False nothrow:True cmd: ['bash', '-c', 'kill -9 2511'] (cluster.py:2173, exec_in_container) 2026-04-30 16:08:48 [ 389 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestdropislockfree_gw0_node_1', 'bash', '-c', 'kill -9 2511'] (cluster.py:113, run_and_check) _______________ test_query_is_permanent[detach-part-TxRollback] ________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxRollback' permanent = ("ALTER TABLE {table} DETACH PARTITION '20221001';", 49) exclusive_table = 'table_test_query_is_permanent_detach_part_TxRollback_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:08:51 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_detach_part_TxRollback_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) ______________ test_query_is_permanent[detach-partition-all-NoTx] ______________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'NoTx' permanent = ('ALTER TABLE {table} DETACH PARTITION ALL;', 0) exclusive_table = 'table_test_query_is_permanent_detach_partition_all_NoTx_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:08:52 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_detach_partition_all_NoTx_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) ____________ test_query_is_permanent[detach-partition-all-TxCommit] ____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxCommit' permanent = ('ALTER TABLE {table} DETACH PARTITION ALL;', 0) exclusive_table = 'table_test_query_is_permanent_detach_partition_all_TxCommit_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:08:56 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_detach_partition_all_TxCommit_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) _________ test_query_is_permanent[detach-partition-all-TxNotFinished] __________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxNotFinished' permanent = ('ALTER TABLE {table} DETACH PARTITION ALL;', 0) exclusive_table = 'table_test_query_is_permanent_detach_partition_all_TxNotFinished_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:08:58 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_detach_partition_all_TxNotFinished_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) ___________ test_query_is_permanent[detach-partition-all-TxRollback] ___________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxRollback' permanent = ('ALTER TABLE {table} DETACH PARTITION ALL;', 0) exclusive_table = 'table_test_query_is_permanent_detach_partition_all_TxRollback_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:08:59 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_detach_partition_all_TxRollback_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) ___________________ test_query_is_permanent[drop-part-NoTx] ____________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'NoTx' permanent = ("ALTER TABLE {table} DROP PART '20220901_1_1_0';", 49) exclusive_table = 'table_test_query_is_permanent_drop_part_NoTx_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:09:00 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_drop_part_NoTx_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) _________________ test_query_is_permanent[drop-part-TxCommit] __________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxCommit' permanent = ("ALTER TABLE {table} DROP PART '20220901_1_1_0';", 49) exclusive_table = 'table_test_query_is_permanent_drop_part_TxCommit_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:09:02 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_drop_part_TxCommit_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) _______________ test_query_is_permanent[drop-part-TxNotFinished] _______________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxNotFinished' permanent = ("ALTER TABLE {table} DROP PART '20220901_1_1_0';", 49) exclusive_table = 'table_test_query_is_permanent_drop_part_TxNotFinished_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:09:03 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_drop_part_TxNotFinished_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) ________________ test_query_is_permanent[drop-part-TxRollback] _________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxRollback' permanent = ("ALTER TABLE {table} DROP PART '20220901_1_1_0';", 49) exclusive_table = 'table_test_query_is_permanent_drop_part_TxRollback_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:09:04 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_drop_part_TxRollback_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) ____________________ test_query_is_permanent[truncate-NoTx] ____________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'NoTx', permanent = ('TRUNCATE TABLE {table};', 0) exclusive_table = 'table_test_query_is_permanent_truncate_NoTx_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:09:05 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_truncate_NoTx_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) __________________ test_query_is_permanent[truncate-TxCommit] __________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxCommit', permanent = ('TRUNCATE TABLE {table};', 0) exclusive_table = 'table_test_query_is_permanent_truncate_TxCommit_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:09:07 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_truncate_TxCommit_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) _______________ test_query_is_permanent[truncate-TxNotFinished] ________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxNotFinished', permanent = ('TRUNCATE TABLE {table};', 0) exclusive_table = 'table_test_query_is_permanent_truncate_TxNotFinished_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:09:08 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_truncate_TxNotFinished_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) _________________ test_query_is_permanent[truncate-TxRollback] _________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 transaction = 'TxRollback', permanent = ('TRUNCATE TABLE {table};', 0) exclusive_table = 'table_test_query_is_permanent_truncate_TxRollback_' @pytest.mark.parametrize( "transaction", ["NoTx", "TxCommit", "TxRollback", "TxNotFinished"] ) @pytest.mark.parametrize( "permanent", PERMANENT_QUERIES.values(), ids=PERMANENT_QUERIES.keys() ) def test_query_is_permanent(transaction, permanent, exclusive_table): > node.query( f""" CREATE TABLE {exclusive_table} ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; """ ) test_drop_is_lock_free/test.py:157: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection refused (172.16.5.2:9000). (NETWORK_ERROR) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-30 16:09:08 [ 389 ] DEBUG : Executing query CREATE TABLE table_test_query_is_permanent_truncate_TxRollback_ ( a Int64, date Date ) Engine=MergeTree PARTITION BY date ORDER BY a; on node (cluster.py:3602, query) ---------------------------- Captured log teardown ----------------------------- 2026-04-30 16:09:11 [ 389 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_drop_is_lock_free/_instances_0_gw0/.env', '--project-name', 'roottestdropislockfree_gw0', '--file', '/ClickHouse/tests/integration/test_drop_is_lock_free/_instances_0_gw0/node/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:113, run_and_check) 2026-04-30 16:09:17 [ 389 ] DEBUG : Stderr:Stopping roottestdropislockfree_gw0_node_1 ... (cluster.py:123, run_and_check) 2026-04-30 16:09:17 [ 389 ] DEBUG : Stderr:Stopping roottestdropislockfree_gw0_node_1 ... done (cluster.py:123, run_and_check) 2026-04-30 16:09:17 [ 389 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_drop_is_lock_free/_instances_0_gw0/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_drop_is_lock_free/_instances_0_gw0/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-04-30 16:09:17 [ 389 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_drop_is_lock_free/_instances_0_gw0/.env', '--project-name', 'roottestdropislockfree_gw0', '--file', '/ClickHouse/tests/integration/test_drop_is_lock_free/_instances_0_gw0/node/docker-compose.yml', 'down', '--volumes'] (cluster.py:113, run_and_check) 2026-04-30 16:09:23 [ 389 ] DEBUG : Stderr:Removing roottestdropislockfree_gw0_node_1 ... (cluster.py:123, run_and_check) 2026-04-30 16:09:23 [ 389 ] DEBUG : Stderr:Removing roottestdropislockfree_gw0_node_1 ... done (cluster.py:123, run_and_check) 2026-04-30 16:09:23 [ 389 ] DEBUG : Stderr:Removing network roottestdropislockfree_gw0_default (cluster.py:123, run_and_check) 2026-04-30 16:09:23 [ 389 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-04-30 16:09:23 [ 389 ] DEBUG : Docker networks for project roottestdropislockfree_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-04-30 16:09:23 [ 389 ] DEBUG : Docker containers for project roottestdropislockfree_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-04-30 16:09:24 [ 389 ] DEBUG : Docker volumes for project roottestdropislockfree_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-04-30 16:09:24 [ 389 ] DEBUG : Command:docker container list --all --filter name='^/roottestdropislockfree_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-04-30 16:09:24 [ 389 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-04-30 16:09:24 [ 389 ] DEBUG : No running containers for project: roottestdropislockfree_gw0 (cluster.py:904, cleanup) 2026-04-30 16:09:24 [ 389 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-04-30 16:09:24 [ 389 ] DEBUG : Trying to prune unused images... (cluster.py:926, cleanup) 2026-04-30 16:09:24 [ 389 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:113, run_and_check) 2026-04-30 16:09:24 [ 389 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:121, run_and_check) 2026-04-30 16:09:24 [ 389 ] DEBUG : Images pruned (cluster.py:929, cleanup) 2026-04-30 16:09:24 [ 389 ] DEBUG : Trying to prune unused volumes... (cluster.py:935, cleanup) 2026-04-30 16:09:24 [ 389 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-04-30 16:09:24 [ 389 ] DEBUG : Stdout:3 (cluster.py:121, run_and_check) ______________________________ test_memory_usage _______________________________ [gw5] linux -- Python 3.10.12 /usr/bin/python3 def test_memory_usage(): node.query("CREATE USER IF NOT EXISTS A") node.query("GRANT ALL ON *.* TO A") node.query( "CREATE TABLE async_table(data Array(UInt64)) ENGINE=MergeTree() ORDER BY data" ) node.get_query_request("SELECT count() FROM system.numbers", user="A") INSERT_QUERY = "INSERT INTO async_table SETTINGS async_insert=1, wait_for_async_insert=1,async_insert_max_data_size=150000000 VALUES ({})" for iter in range(10): values = list(range(iter * 5000000, (iter + 1) * 5000000)) > node.query(INSERT_QUERY.format(values), user="A") test_async_insert_memory/test.py:39: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() helpers/client.py:221: in get_answer self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) /usr/lib/python3.10/subprocess.py:1209: in wait return self._wait(timeout=timeout) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 600 def _wait(self, timeout): """Internal implementation of wait() on POSIX.""" if self.returncode is not None: return self.returncode if timeout is not None: endtime = _time() + timeout # Enter a busy loop if we have a timeout. This busy loop was # cribbed from Lib/threading.py in Thread.wait() at r71065. delay = 0.0005 # 500 us -> initial delay of 1 ms while True: if self._waitpid_lock.acquire(False): try: if self.returncode is not None: break # Another thread waited. (pid, sts) = self._try_wait(os.WNOHANG) assert pid == self.pid or pid == 0 if pid == self.pid: self._handle_exitstatus(sts) break finally: self._waitpid_lock.release() remaining = self._remaining_time(endtime) if remaining <= 0: raise TimeoutExpired(self.args, timeout) delay = min(delay * 2, remaining, .05) > time.sleep(delay) E Failed: Timeout >900.0s /usr/lib/python3.10/subprocess.py:1953: Failed ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ------------------------------ Captured log setup ------------------------------ 2026-04-30 15:57:34 [ 404 ] INFO : Running tests in /ClickHouse/tests/integration/test_async_insert_memory/test.py (cluster.py:2788, start) 2026-04-30 15:57:35 [ 404 ] DEBUG : Cluster start called. is_up=False (cluster.py:2795, start) 2026-04-30 15:57:35 [ 404 ] DEBUG : Docker networks for project roottestasyncinsertmemory_gw5 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-04-30 15:57:35 [ 404 ] DEBUG : Docker containers for project roottestasyncinsertmemory_gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-04-30 15:57:35 [ 404 ] DEBUG : Docker volumes for project roottestasyncinsertmemory_gw5 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-04-30 15:57:35 [ 404 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-04-30 15:57:36 [ 404 ] DEBUG : Docker networks for project roottestasyncinsertmemory_gw5 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-04-30 15:57:36 [ 404 ] DEBUG : Docker containers for project roottestasyncinsertmemory_gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-04-30 15:57:36 [ 404 ] DEBUG : Docker volumes for project roottestasyncinsertmemory_gw5 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-04-30 15:57:36 [ 404 ] DEBUG : Command:docker container list --all --filter name='^/roottestasyncinsertmemory_gw5_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-04-30 15:57:36 [ 404 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-04-30 15:57:36 [ 404 ] DEBUG : No running containers for project: roottestasyncinsertmemory_gw5 (cluster.py:904, cleanup) 2026-04-30 15:57:36 [ 404 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-04-30 15:57:37 [ 404 ] DEBUG : Trying to prune unused images... (cluster.py:926, cleanup) 2026-04-30 15:57:37 [ 404 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:113, run_and_check) 2026-04-30 15:57:37 [ 404 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:121, run_and_check) 2026-04-30 15:57:37 [ 404 ] DEBUG : Images pruned (cluster.py:929, cleanup) 2026-04-30 15:57:37 [ 404 ] DEBUG : Trying to prune unused volumes... (cluster.py:935, cleanup) 2026-04-30 15:57:37 [ 404 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-04-30 15:57:37 [ 404 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-04-30 15:57:37 [ 404 ] DEBUG : Setup directory for instance: node (cluster.py:2808, start) 2026-04-30 15:57:37 [ 404 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4534, create_dir) 2026-04-30 15:57:37 [ 404 ] DEBUG : Create directory for common tests configuration (cluster.py:4539, create_dir) 2026-04-30 15:57:37 [ 404 ] DEBUG : Copy common configuration from helpers (cluster.py:4559, create_dir) 2026-04-30 15:57:37 [ 404 ] DEBUG : Generate and write macros file (cluster.py:4602, create_dir) 2026-04-30 15:57:37 [ 404 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/configs/config.d (cluster.py:4632, create_dir) 2026-04-30 15:57:37 [ 404 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/database (cluster.py:4649, create_dir) 2026-04-30 15:57:37 [ 404 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/logs (cluster.py:4660, create_dir) 2026-04-30 15:57:37 [ 404 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4746, create_dir) 2026-04-30 15:57:37 [ 404 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/.env (cluster.py:86, _create_env_file) 2026-04-30 15:57:37 [ 404 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-30 15:57:37 [ 404 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-30 15:57:37 [ 404 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-30 15:57:37 [ 404 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-30 15:57:37 [ 404 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 824 (connectionpool.py:547, _make_request) 2026-04-30 15:57:37 [ 404 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/.env', '--project-name', 'roottestasyncinsertmemory_gw5', '--file', '/ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-04-30 15:58:23 [ 404 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-04-30 15:58:23 [ 404 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-04-30 15:58:23 [ 404 ] DEBUG : Stderr:Pulling node ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-04-30 15:58:23 [ 404 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-04-30 15:58:23 [ 404 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-04-30 15:58:23 [ 404 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker-compose --env-file /ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/.env --project-name roottestasyncinsertmemory_gw5 --file /ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/docker-compose.yml up -d --no-recreate') (cluster.py:3146, start) 2026-04-30 15:58:23 [ 404 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/.env', '--project-name', 'roottestasyncinsertmemory_gw5', '--file', '/ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/docker-compose.yml', 'up', '-d', '--no-recreate'] (cluster.py:113, run_and_check) 2026-04-30 15:58:33 [ 404 ] DEBUG : Stderr:Creating network "roottestasyncinsertmemory_gw5_default" with the default driver (cluster.py:123, run_and_check) 2026-04-30 15:58:33 [ 404 ] DEBUG : Stderr:Creating roottestasyncinsertmemory_gw5_node_1 ... (cluster.py:123, run_and_check) 2026-04-30 15:58:33 [ 404 ] DEBUG : Stderr:Creating roottestasyncinsertmemory_gw5_node_1 ... done (cluster.py:123, run_and_check) 2026-04-30 15:58:33 [ 404 ] DEBUG : ClickHouse instance created (cluster.py:3154, start) 2026-04-30 15:58:33 [ 404 ] DEBUG : get_instance_ip instance_name=node (cluster.py:2135, get_instance_ip) 2026-04-30 15:58:33 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestasyncinsertmemory_gw5_node_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:33 [ 404 ] DEBUG : Waiting for ClickHouse start in node, ip: 172.16.11.2... (cluster.py:3161, start) 2026-04-30 15:58:33 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestasyncinsertmemory_gw5_node_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:33 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:33 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:33 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:33 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:33 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:33 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:33 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:34 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:34 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:34 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:34 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:34 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:34 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:34 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:35 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:35 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:35 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:35 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:35 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:35 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:35 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:35 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:36 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:36 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:36 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:36 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:36 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:36 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:36 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:37 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:37 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:37 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:37 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:37 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:37 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:37 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:38 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:38 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:38 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:38 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:38 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:38 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:38 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:38 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:39 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:39 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:39 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:39 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:39 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:40 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:40 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:40 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:40 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:40 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:40 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:40 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:40 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:41 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:41 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:41 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:41 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:41 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:41 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:41 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:42 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:42 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:42 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:42 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:42 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:42 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:42 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:42 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:43 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:43 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:43 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:43 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:43 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:43 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:43 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:44 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:44 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:44 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:44 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:44 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:44 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:44 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:45 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:45 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:45 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:45 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:45 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:45 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:45 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:46 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:46 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:46 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:46 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:46 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:46 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:46 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:46 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:47 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:47 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:47 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:47 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:47 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:47 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:47 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:47 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:47 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:48 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:48 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:48 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:48 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:48 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:48 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:48 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:49 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:49 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:49 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:49 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:49 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:49 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:49 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:49 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:50 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:50 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:50 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:50 [ 404 ] DEBUG : http://localhost:None "GET /v1.42/containers/5159ca7a63b985655d7945c110d0eff501c6805a78ddc5e9fbf37d489dbca838/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-30 15:58:50 [ 404 ] DEBUG : ClickHouse node started (cluster.py:3165, start) ----------------------------- Captured stderr call ----------------------------- ~~~~~~~~~~~~~~~~~~~~~ Stack of (140556430530112) ~~~~~~~~~~~~~~~~~~~~~ File "/usr/local/lib/python3.10/dist-packages/execnet/gateway_base.py", line 411, in _perform_spawn reply.run() File "/usr/local/lib/python3.10/dist-packages/execnet/gateway_base.py", line 341, in run self._result = func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/execnet/gateway_base.py", line 1160, in _thread_receiver msg = Message.from_io(io) File "/usr/local/lib/python3.10/dist-packages/execnet/gateway_base.py", line 567, in from_io header = io.read(9) # type 1, channel 4, payload 4 File "/usr/local/lib/python3.10/dist-packages/execnet/gateway_base.py", line 534, in read data = self._read(numbytes - len(buf)) ------------------------------ Captured log call ------------------------------- 2026-04-30 15:58:50 [ 404 ] DEBUG : Executing query CREATE USER IF NOT EXISTS A on node (cluster.py:3602, query) 2026-04-30 15:58:52 [ 404 ] DEBUG : Executing query GRANT ALL ON *.* TO A on node (cluster.py:3602, query) 2026-04-30 15:58:54 [ 404 ] DEBUG : Executing query CREATE TABLE async_table(data Array(UInt64)) ENGINE=MergeTree() ORDER BY data on node (cluster.py:3602, query) 2026-04-30 15:58:56 [ 404 ] DEBUG : Executing query SELECT count() FROM system.numbers on node (cluster.py:3667, get_query_request) 2026-04-30 15:58:59 [ 404 ] DEBUG : Executing query INSERT INTO async_table SETTINGS async_insert=1, wait_for_async_insert=1,async_insert_max_data_size=150000000 VALUES ([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 1 on node (cluster.py:3602, query) 2026-04-30 16:00:32 [ 404 ] DEBUG : Executing query INSERT INTO async_table SETTINGS async_insert=1, wait_for_async_insert=1,async_insert_max_data_size=150000000 VALUES ([5000000, 5000001, 5000002, 5000003, 5000004, 5000005, 5000006, 5000007, 5000008, 5000009, 5000010, 5000011, 5000012, 5000013, 5000014, 5000015, 5000016, 5000017, 5000018, 5000019, 5000020, 5000021, 5000022, 5000023, 5000024, 5000025, 5000026, 5000027, 5000028, 5000029, 5000030, 5000031, 5000032, 5000033, 5000034, 5000035, 5000036, 5000037, 5000038, 5000039, 5000040, 5000041, 5000042, 5000043, 5000044, 5000045, 5000046, 5000047, 5000048, 5000049, 5000050, 5000051, 5000052, 5000053, 5000054, 5000055, 5000056, 5000057, 5000058, 5000059, 5000060, 5000061, 5000062, 5000063, 5000064, 5000065, 5000066, 5000067, 5000068, 5000069, 5000070, 5000071, 5000072, 5000073, 5000074, 5000075, 5000076, 5000077, 5000078, 5000079, 5000080, 5000081, 5000082, 5000083, 5000084, 5000085, 5000086, 5000087, 5000088, 5000089, 5000090, 5000091, 5000092, 5000093, 5000094, 5000095, 5000096, 5000097, on node (cluster.py:3602, query) 2026-04-30 16:02:09 [ 404 ] DEBUG : Executing query INSERT INTO async_table SETTINGS async_insert=1, wait_for_async_insert=1,async_insert_max_data_size=150000000 VALUES ([10000000, 10000001, 10000002, 10000003, 10000004, 10000005, 10000006, 10000007, 10000008, 10000009, 10000010, 10000011, 10000012, 10000013, 10000014, 10000015, 10000016, 10000017, 10000018, 10000019, 10000020, 10000021, 10000022, 10000023, 10000024, 10000025, 10000026, 10000027, 10000028, 10000029, 10000030, 10000031, 10000032, 10000033, 10000034, 10000035, 10000036, 10000037, 10000038, 10000039, 10000040, 10000041, 10000042, 10000043, 10000044, 10000045, 10000046, 10000047, 10000048, 10000049, 10000050, 10000051, 10000052, 10000053, 10000054, 10000055, 10000056, 10000057, 10000058, 10000059, 10000060, 10000061, 10000062, 10000063, 10000064, 10000065, 10000066, 10000067, 10000068, 10000069, 10000070, 10000071, 10000072, 10000073, 10000074, 10000075, 10000076, 10000077, 10000078, 10000079, 10000080, 10000081, 10000082, 10000083, 10000084, 10000085, 10000086, 10000087, 1 on node (cluster.py:3602, query) 2026-04-30 16:04:24 [ 404 ] DEBUG : Executing query INSERT INTO async_table SETTINGS async_insert=1, wait_for_async_insert=1,async_insert_max_data_size=150000000 VALUES ([15000000, 15000001, 15000002, 15000003, 15000004, 15000005, 15000006, 15000007, 15000008, 15000009, 15000010, 15000011, 15000012, 15000013, 15000014, 15000015, 15000016, 15000017, 15000018, 15000019, 15000020, 15000021, 15000022, 15000023, 15000024, 15000025, 15000026, 15000027, 15000028, 15000029, 15000030, 15000031, 15000032, 15000033, 15000034, 15000035, 15000036, 15000037, 15000038, 15000039, 15000040, 15000041, 15000042, 15000043, 15000044, 15000045, 15000046, 15000047, 15000048, 15000049, 15000050, 15000051, 15000052, 15000053, 15000054, 15000055, 15000056, 15000057, 15000058, 15000059, 15000060, 15000061, 15000062, 15000063, 15000064, 15000065, 15000066, 15000067, 15000068, 15000069, 15000070, 15000071, 15000072, 15000073, 15000074, 15000075, 15000076, 15000077, 15000078, 15000079, 15000080, 15000081, 15000082, 15000083, 15000084, 15000085, 15000086, 15000087, 1 on node (cluster.py:3602, query) 2026-04-30 16:06:38 [ 404 ] DEBUG : Executing query INSERT INTO async_table SETTINGS async_insert=1, wait_for_async_insert=1,async_insert_max_data_size=150000000 VALUES ([20000000, 20000001, 20000002, 20000003, 20000004, 20000005, 20000006, 20000007, 20000008, 20000009, 20000010, 20000011, 20000012, 20000013, 20000014, 20000015, 20000016, 20000017, 20000018, 20000019, 20000020, 20000021, 20000022, 20000023, 20000024, 20000025, 20000026, 20000027, 20000028, 20000029, 20000030, 20000031, 20000032, 20000033, 20000034, 20000035, 20000036, 20000037, 20000038, 20000039, 20000040, 20000041, 20000042, 20000043, 20000044, 20000045, 20000046, 20000047, 20000048, 20000049, 20000050, 20000051, 20000052, 20000053, 20000054, 20000055, 20000056, 20000057, 20000058, 20000059, 20000060, 20000061, 20000062, 20000063, 20000064, 20000065, 20000066, 20000067, 20000068, 20000069, 20000070, 20000071, 20000072, 20000073, 20000074, 20000075, 20000076, 20000077, 20000078, 20000079, 20000080, 20000081, 20000082, 20000083, 20000084, 20000085, 20000086, 20000087, 2 on node (cluster.py:3602, query) 2026-04-30 16:07:49 [ 404 ] DEBUG : Executing query INSERT INTO async_table SETTINGS async_insert=1, wait_for_async_insert=1,async_insert_max_data_size=150000000 VALUES ([25000000, 25000001, 25000002, 25000003, 25000004, 25000005, 25000006, 25000007, 25000008, 25000009, 25000010, 25000011, 25000012, 25000013, 25000014, 25000015, 25000016, 25000017, 25000018, 25000019, 25000020, 25000021, 25000022, 25000023, 25000024, 25000025, 25000026, 25000027, 25000028, 25000029, 25000030, 25000031, 25000032, 25000033, 25000034, 25000035, 25000036, 25000037, 25000038, 25000039, 25000040, 25000041, 25000042, 25000043, 25000044, 25000045, 25000046, 25000047, 25000048, 25000049, 25000050, 25000051, 25000052, 25000053, 25000054, 25000055, 25000056, 25000057, 25000058, 25000059, 25000060, 25000061, 25000062, 25000063, 25000064, 25000065, 25000066, 25000067, 25000068, 25000069, 25000070, 25000071, 25000072, 25000073, 25000074, 25000075, 25000076, 25000077, 25000078, 25000079, 25000080, 25000081, 25000082, 25000083, 25000084, 25000085, 25000086, 25000087, 2 on node (cluster.py:3602, query) 2026-04-30 16:09:13 [ 404 ] DEBUG : Executing query INSERT INTO async_table SETTINGS async_insert=1, wait_for_async_insert=1,async_insert_max_data_size=150000000 VALUES ([30000000, 30000001, 30000002, 30000003, 30000004, 30000005, 30000006, 30000007, 30000008, 30000009, 30000010, 30000011, 30000012, 30000013, 30000014, 30000015, 30000016, 30000017, 30000018, 30000019, 30000020, 30000021, 30000022, 30000023, 30000024, 30000025, 30000026, 30000027, 30000028, 30000029, 30000030, 30000031, 30000032, 30000033, 30000034, 30000035, 30000036, 30000037, 30000038, 30000039, 30000040, 30000041, 30000042, 30000043, 30000044, 30000045, 30000046, 30000047, 30000048, 30000049, 30000050, 30000051, 30000052, 30000053, 30000054, 30000055, 30000056, 30000057, 30000058, 30000059, 30000060, 30000061, 30000062, 30000063, 30000064, 30000065, 30000066, 30000067, 30000068, 30000069, 30000070, 30000071, 30000072, 30000073, 30000074, 30000075, 30000076, 30000077, 30000078, 30000079, 30000080, 30000081, 30000082, 30000083, 30000084, 30000085, 30000086, 30000087, 3 on node (cluster.py:3602, query) 2026-04-30 16:10:52 [ 404 ] DEBUG : Executing query INSERT INTO async_table SETTINGS async_insert=1, wait_for_async_insert=1,async_insert_max_data_size=150000000 VALUES ([35000000, 35000001, 35000002, 35000003, 35000004, 35000005, 35000006, 35000007, 35000008, 35000009, 35000010, 35000011, 35000012, 35000013, 35000014, 35000015, 35000016, 35000017, 35000018, 35000019, 35000020, 35000021, 35000022, 35000023, 35000024, 35000025, 35000026, 35000027, 35000028, 35000029, 35000030, 35000031, 35000032, 35000033, 35000034, 35000035, 35000036, 35000037, 35000038, 35000039, 35000040, 35000041, 35000042, 35000043, 35000044, 35000045, 35000046, 35000047, 35000048, 35000049, 35000050, 35000051, 35000052, 35000053, 35000054, 35000055, 35000056, 35000057, 35000058, 35000059, 35000060, 35000061, 35000062, 35000063, 35000064, 35000065, 35000066, 35000067, 35000068, 35000069, 35000070, 35000071, 35000072, 35000073, 35000074, 35000075, 35000076, 35000077, 35000078, 35000079, 35000080, 35000081, 35000082, 35000083, 35000084, 35000085, 35000086, 35000087, 3 on node (cluster.py:3602, query) ---------------------------- Captured log teardown ----------------------------- 2026-04-30 16:12:40 [ 404 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/.env', '--project-name', 'roottestasyncinsertmemory_gw5', '--file', '/ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:113, run_and_check) 2026-04-30 16:13:04 [ 404 ] DEBUG : Stderr:Stopping roottestasyncinsertmemory_gw5_node_1 ... (cluster.py:123, run_and_check) 2026-04-30 16:13:04 [ 404 ] DEBUG : Stderr:Stopping roottestasyncinsertmemory_gw5_node_1 ... done (cluster.py:123, run_and_check) 2026-04-30 16:13:04 [ 404 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-04-30 16:13:04 [ 404 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/.env', '--project-name', 'roottestasyncinsertmemory_gw5', '--file', '/ClickHouse/tests/integration/test_async_insert_memory/_instances_0_gw5/node/docker-compose.yml', 'down', '--volumes'] (cluster.py:113, run_and_check) 2026-04-30 16:13:09 [ 404 ] DEBUG : Stderr:Removing roottestasyncinsertmemory_gw5_node_1 ... (cluster.py:123, run_and_check) 2026-04-30 16:13:09 [ 404 ] DEBUG : Stderr:Removing roottestasyncinsertmemory_gw5_node_1 ... done (cluster.py:123, run_and_check) 2026-04-30 16:13:09 [ 404 ] DEBUG : Stderr:Removing network roottestasyncinsertmemory_gw5_default (cluster.py:123, run_and_check) 2026-04-30 16:13:09 [ 404 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-04-30 16:13:10 [ 404 ] DEBUG : Docker networks for project roottestasyncinsertmemory_gw5 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-04-30 16:13:10 [ 404 ] DEBUG : Docker containers for project roottestasyncinsertmemory_gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-04-30 16:13:10 [ 404 ] DEBUG : Docker volumes for project roottestasyncinsertmemory_gw5 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-04-30 16:13:10 [ 404 ] DEBUG : Command:docker container list --all --filter name='^/roottestasyncinsertmemory_gw5_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-04-30 16:13:11 [ 404 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-04-30 16:13:11 [ 404 ] DEBUG : No running containers for project: roottestasyncinsertmemory_gw5 (cluster.py:904, cleanup) 2026-04-30 16:13:11 [ 404 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-04-30 16:13:11 [ 404 ] DEBUG : Trying to prune unused images... (cluster.py:926, cleanup) 2026-04-30 16:13:11 [ 404 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:113, run_and_check) 2026-04-30 16:13:11 [ 404 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:121, run_and_check) 2026-04-30 16:13:11 [ 404 ] DEBUG : Images pruned (cluster.py:929, cleanup) 2026-04-30 16:13:11 [ 404 ] DEBUG : Trying to prune unused volumes... (cluster.py:935, cleanup) 2026-04-30 16:13:11 [ 404 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-04-30 16:13:11 [ 404 ] DEBUG : Stdout:3 (cluster.py:121, run_and_check) ============================== slowest durations =============================== 824.42s call test_async_insert_memory/test.py::test_memory_usage 552.34s call test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic 466.53s call test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated 405.81s call test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxNotFinished] 291.12s call test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_simple[cache] 290.57s call test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas 284.28s call test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool 266.37s call test_consistant_parts_after_move_partition/test.py::test_consistent_part_after_move_partition 254.02s call test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_complex[complex_key_cache] 250.48s setup test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic 233.90s setup test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3] 149.79s setup test_dictionaries_config_reload/test.py::test 134.38s setup test_access_control_on_cluster/test.py::test_access_control_on_cluster 108.39s call test_backup_restore_s3/test.py::test_backup_to_s3_different_credentials 101.22s setup test_executable_user_defined_function/test.py::test_executable_function_always_error_python 98.83s setup test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary0-instances_to_check0] 97.40s setup test_failed_async_inserts/test.py::test_failed_async_inserts 95.01s call test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxCommit] 89.42s setup test_groupBitmapAnd_on_distributed/test.py::test_groupBitmapAnd_on_distributed_table 88.02s setup test_consistant_parts_after_move_partition/test.py::test_consistent_part_after_move_partition 87.73s setup test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool 86.69s call test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-NoTx] 83.18s call test_backup_restore_s3/test.py::test_backup_to_s3_multipart 79.89s setup test_extreme_deduplication/test.py::test_deduplication_window_in_seconds 78.31s call test_backup_restore_s3/test.py::test_backup_to_s3_native_copy_multipart 78.05s setup test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PART-2020-08-28-20200828_0_0_0] 78.04s setup test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Atomic] 75.66s setup test_async_insert_memory/test.py::test_memory_usage 73.88s setup test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field 72.37s setup test_dictionaries_access/test.py::test_create 72.34s setup test_dictionaries_with_invalid_structure/test.py::test_dictGet_func_for_invalid_dictionary 68.94s setup test_drop_is_lock_free/test.py::test_query_is_lock_free[detach part] 68.93s setup test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_complex[complex_key_cache] 68.49s setup test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability 64.67s setup test_accept_invalid_certificate/test.py::test_accept 62.72s setup test_compression_nested_columns/test.py::test_nested_compression_codec 62.62s setup test_config_reloader_interval/test.py::test_reload_config 55.26s call test_backup_restore_s3/test.py::test_user_specific_auth 51.58s setup test_backup_log/test.py::test_backup_log 45.96s teardown test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary3-instances_to_check3] 45.76s teardown test_consistant_parts_after_move_partition/test.py::test_consistent_part_after_move_partition 44.01s call test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_plain_rewritable] 43.73s teardown test_access_control_on_cluster/test.py::test_grant_current_database_on_cluster 42.49s call test_backup_restore_s3/test.py::test_backup_to_s3 40.51s teardown test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Ordinary] 39.44s call test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Ordinary] 38.43s call test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Atomic] 37.57s call test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field 36.06s call test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary3-instances_to_check3] 35.77s call test_access_control_on_cluster/test.py::test_access_control_on_cluster 35.30s call test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary2-instances_to_check2] 34.99s teardown test_async_insert_memory/test.py::test_memory_usage 34.63s teardown test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_simple[cache] 34.31s teardown test_extreme_deduplication/test.py::test_deduplication_window_in_seconds 34.30s teardown test_compression_nested_columns/test.py::test_nested_compression_codec 34.26s call test_dictionaries_config_reload/test.py::test 33.19s teardown test_dictionaries_access/test.py::test_drop 32.36s call test_drop_is_lock_free/test.py::test_query_is_lock_free[truncate] 32.34s call test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3] 31.43s teardown test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PARTITION-2020-08-27-2020-08-27] 31.37s teardown test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool 30.75s call test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3_plain] 30.52s call test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_s3_plain] 30.47s call test_compression_nested_columns/test.py::test_nested_compression_codec 30.07s call test_drop_is_lock_free/test.py::test_query_is_lock_free[detach table] 30.01s call test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool 29.35s teardown test_backup_restore_s3/test.py::test_user_specific_auth 29.32s call test_extreme_deduplication/test.py::test_deduplication_window_in_seconds 29.21s call test_backup_restore_s3/test.py::test_incremental_backup_append_table_def 28.09s teardown test_groupBitmapAnd_on_distributed/test.py::test_groupBitmapAnd_on_distributed_table 27.89s call test_executable_user_defined_function/test.py::test_executable_function_sum_json_python 27.88s call test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-False-False] 26.91s call test_drop_is_lock_free/test.py::test_query_is_lock_free[detach part] 26.61s teardown test_failed_async_inserts/test.py::test_failed_async_inserts 26.57s teardown test_dictionaries_with_invalid_structure/test.py::test_select_from_system_dictionaries_with_invalid_dictionary 26.42s call test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_local] 26.39s call test_backup_log/test.py::test_backup_log 25.79s call test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3_other_bucket] 25.69s call test_drop_is_lock_free/test.py::test_query_is_lock_free[drop part] 24.16s call test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3] 24.08s call test_dictionaries_access/test.py::test_create 23.67s call test_executable_user_defined_function/test.py::test_executable_function_always_error_python 23.62s call test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_local] 23.02s teardown test_accept_invalid_certificate/test.py::test_strict_reject_with_config 22.74s call test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability 22.39s call test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_other_bucket] 21.68s call test_access_control_on_cluster/test.py::test_grant_current_database_on_cluster 21.67s call test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-False] 21.66s teardown test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability 20.76s call test_backup_restore_s3/test.py::test_backup_with_fs_cache[False-True-False] 20.10s call test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3] 19.69s call test_backup_restore_s3/test.py::test_backup_to_s3_named_collection 19.59s call test_backup_restore_s3/test.py::test_backup_to_tar 19.37s call test_backup_restore_s3/test.py::test_backup_to_tar_lzma 18.95s call test_backup_restore_s3/test.py::test_backup_to_tar_xz 18.84s call test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PART-2020-08-28-20200828_0_0_0] 18.55s call test_backup_restore_s3/test.py::test_backup_to_tar_zst 18.46s call test_executable_user_defined_function/test.py::test_executable_function_slow_python 18.41s teardown test_config_reloader_interval/test.py::test_reload_config 18.13s teardown test_executable_user_defined_function/test.py::test_executable_function_sum_python 18.12s call test_backup_restore_s3/test.py::test_backup_to_tar_bz2 17.55s call test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-True] 17.36s call test_backup_restore_s3/test.py::test_backup_to_zip 17.22s call test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary1-instances_to_check1] 16.98s teardown test_dictionaries_config_reload/test.py::test 16.97s call test_backup_restore_s3/test.py::test_backup_to_tar_gz 16.87s call test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary0-instances_to_check0] 16.21s call test_failed_async_inserts/test.py::test_failed_async_inserts 15.05s teardown test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic 14.65s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxRollback] 13.89s call test_dictionaries_access/test.py::test_dictget 13.09s call test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python 12.54s call test_groupBitmapAnd_on_distributed/test.py::test_groupBitmapAnd_on_distributed_table 12.18s call test_dictionaries_access/test.py::test_drop 12.03s call test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python 11.95s call test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash 11.49s call test_executable_user_defined_function/test.py::test_executable_function_parameter_python 10.71s call test_executable_user_defined_function/test.py::test_executable_function_python 10.25s call test_executable_user_defined_function/test.py::test_executable_function_sum_python 9.66s call test_access_control_on_cluster/test.py::test_grant_all_on_cluster 9.10s call test_executable_user_defined_function/test.py::test_executable_function_argument_python 8.69s call test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PARTITION-2020-08-27-2020-08-27] 8.64s call test_executable_user_defined_function/test.py::test_executable_function_signalled_python 7.91s call test_executable_user_defined_function/test.py::test_executable_function_bash 7.14s teardown test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field 6.13s call test_dictionaries_with_invalid_structure/test.py::test_select_from_system_dictionaries_with_invalid_dictionary 5.01s teardown test_backup_log/test.py::test_backup_log 4.30s teardown test_dictionaries_access/test.py::test_create 3.59s call test_config_reloader_interval/test.py::test_reload_config 3.31s call test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-NoTx] 2.80s teardown test_dictionaries_access/test.py::test_dictget 2.53s teardown test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary0-instances_to_check0] 2.06s call test_dictionaries_with_invalid_structure/test.py::test_dictGet_func_for_invalid_dictionary 1.84s call test_accept_invalid_certificate/test.py::test_accept 1.78s call test_accept_invalid_certificate/test.py::test_strict_reject 1.69s teardown test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary2-instances_to_check2] 1.39s call test_accept_invalid_certificate/test.py::test_connection_accept 1.38s call test_accept_invalid_certificate/test.py::test_strict_reject_with_config 1.36s call test_accept_invalid_certificate/test.py::test_default 1.33s call test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxCommit] 1.31s call test_accept_invalid_certificate/test.py::test_strict_connection_reject 1.28s call test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxCommit] 1.24s teardown test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary1-instances_to_check1] 0.91s call test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxRollback] 0.76s call test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-NoTx] 0.71s call test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-NoTx] 0.71s call test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxNotFinished] 0.68s call test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxNotFinished] 0.65s call test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxRollback] 0.63s call test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxRollback] 0.58s call test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxRollback] 0.55s call test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxCommit] 0.49s call test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxNotFinished] 0.09s teardown test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool 0.06s setup test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_local] 0.05s setup test_executable_user_defined_function/test.py::test_executable_function_slow_python 0.04s setup test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxRollback] 0.03s teardown test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated 0.03s teardown test_executable_user_defined_function/test.py::test_executable_function_always_error_python 0.03s teardown test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Atomic] 0.03s teardown test_drop_is_lock_free/test.py::test_query_is_lock_free[drop part] 0.02s teardown test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3_plain] 0.02s setup test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxCommit] 0.02s setup test_dictionaries_access/test.py::test_drop 0.02s setup test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas 0.02s teardown test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_complex[complex_key_cache] 0.02s teardown test_access_control_on_cluster/test.py::test_access_control_on_cluster 0.02s teardown test_accept_invalid_certificate/test.py::test_accept 0.02s setup test_drop_is_lock_free/test.py::test_query_is_lock_free[detach table] 0.02s setup test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary1-instances_to_check1] 0.01s setup test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_simple[cache] 0.01s teardown test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3] 0.01s teardown test_executable_user_defined_function/test.py::test_executable_function_python 0.01s setup test_accept_invalid_certificate/test.py::test_strict_reject_with_config 0.01s teardown test_accept_invalid_certificate/test.py::test_default 0.01s teardown test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3] 0.01s setup test_drop_is_lock_free/test.py::test_query_is_lock_free[drop part] 0.01s teardown test_backup_restore_s3/test.py::test_backup_to_s3 0.01s teardown test_backup_restore_s3/test.py::test_backup_to_s3_multipart 0.01s setup test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3] 0.01s teardown test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3_other_bucket] 0.01s setup test_accept_invalid_certificate/test.py::test_default 0.01s teardown test_backup_restore_s3/test.py::test_incremental_backup_append_table_def 0.01s setup test_executable_user_defined_function/test.py::test_executable_function_sum_json_python 0.01s setup test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_local] 0.01s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-NoTx] 0.01s teardown test_accept_invalid_certificate/test.py::test_strict_reject 0.01s setup test_backup_restore_s3/test.py::test_backup_with_fs_cache[False-True-False] 0.01s setup test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_s3_plain] 0.01s setup test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-NoTx] 0.01s setup test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-False] 0.01s setup test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-False-False] 0.01s setup test_backup_restore_s3/test.py::test_backup_to_s3_multipart 0.01s teardown test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-False] 0.01s teardown test_backup_restore_s3/test.py::test_backup_to_s3_different_credentials 0.01s setup test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Ordinary] 0.01s setup test_executable_user_defined_function/test.py::test_executable_function_parameter_python 0.01s teardown test_access_control_on_cluster/test.py::test_grant_all_on_cluster 0.01s teardown test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3] 0.01s setup test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated 0.01s setup test_backup_restore_s3/test.py::test_backup_to_tar 0.01s setup test_drop_is_lock_free/test.py::test_query_is_lock_free[truncate] 0.01s setup test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3] 0.01s setup test_backup_restore_s3/test.py::test_incremental_backup_append_table_def 0.01s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxNotFinished] 0.01s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxNotFinished] 0.01s teardown test_executable_user_defined_function/test.py::test_executable_function_argument_python 0.01s teardown test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash 0.01s setup test_executable_user_defined_function/test.py::test_executable_function_bash 0.00s setup test_backup_restore_s3/test.py::test_backup_to_s3_native_copy_multipart 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxNotFinished] 0.00s setup test_backup_restore_s3/test.py::test_backup_to_zip 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_signalled_python 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_plain_rewritable] 0.00s setup test_dictionaries_access/test.py::test_dictget 0.00s setup test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool 0.00s setup test_backup_restore_s3/test.py::test_user_specific_auth 0.00s setup test_backup_restore_s3/test.py::test_backup_to_tar_zst 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_slow_python 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_zip 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_argument_python 0.00s setup test_backup_restore_s3/test.py::test_backup_to_s3_different_credentials 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxNotFinished] 0.00s setup test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3_plain] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxRollback] 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_tar 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_local] 0.00s setup test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3_other_bucket] 0.00s teardown test_backup_restore_s3/test.py::test_backup_with_fs_cache[False-True-False] 0.00s teardown test_dictionaries_with_invalid_structure/test.py::test_dictGet_func_for_invalid_dictionary 0.00s setup test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_other_bucket] 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_signalled_python 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_local] 0.00s teardown test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PART-2020-08-28-20200828_0_0_0] 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_tar_lzma 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_sum_python 0.00s setup test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PARTITION-2020-08-27-2020-08-27] 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_python 0.00s teardown test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-True] 0.00s teardown test_accept_invalid_certificate/test.py::test_connection_accept 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxNotFinished] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-NoTx] 0.00s setup test_access_control_on_cluster/test.py::test_grant_all_on_cluster 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxRollback] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxCommit] 0.00s setup test_backup_restore_s3/test.py::test_backup_to_tar_xz 0.00s teardown test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-False-False] 0.00s setup test_accept_invalid_certificate/test.py::test_connection_accept 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxRollback] 0.00s setup test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-True] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxRollback] 0.00s teardown test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas 0.00s setup test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary2-instances_to_check2] 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxCommit] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-NoTx] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxCommit] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxNotFinished] 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_s3_plain] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxNotFinished] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-NoTx] 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_tar_zst 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxNotFinished] 0.00s setup test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxCommit] 0.00s setup test_access_control_on_cluster/test.py::test_grant_current_database_on_cluster 0.00s setup test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary3-instances_to_check3] 0.00s setup test_dictionaries_with_invalid_structure/test.py::test_select_from_system_dictionaries_with_invalid_dictionary 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_s3_native_copy_multipart 0.00s setup test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_plain_rewritable] 0.00s setup test_backup_restore_s3/test.py::test_backup_to_s3_named_collection 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_tar_gz 0.00s setup test_backup_restore_s3/test.py::test_backup_to_tar_bz2 0.00s setup test_backup_restore_s3/test.py::test_backup_to_tar_gz 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_lock_free[detach table] 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxRollback] 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_other_bucket] 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-NoTx] 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxCommit] 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_lock_free[detach part] 0.00s setup test_backup_restore_s3/test.py::test_backup_to_s3 0.00s setup test_backup_restore_s3/test.py::test_backup_to_tar_lzma 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxRollback] 0.00s setup test_accept_invalid_certificate/test.py::test_strict_reject 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_tar_xz 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_sum_json_python 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_s3_named_collection 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-NoTx] 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-NoTx] 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxCommit] 0.00s setup test_accept_invalid_certificate/test.py::test_strict_connection_reject 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_lock_free[truncate] 0.00s teardown test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxCommit] 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_bash 0.00s teardown test_accept_invalid_certificate/test.py::test_strict_connection_reject 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_parameter_python 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python 0.00s teardown test_backup_restore_s3/test.py::test_backup_to_tar_bz2 =========================== short test summary info ============================ FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxNotFinished] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxRollback] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-NoTx] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxCommit] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxNotFinished] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxRollback] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-NoTx] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxCommit] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxNotFinished] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxRollback] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-NoTx] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxCommit] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxNotFinished] FAILED test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxRollback] FAILED test_async_insert_memory/test.py::test_memory_usage - Failed: Timeout ... PASSED test_accept_invalid_certificate/test.py::test_accept PASSED test_accept_invalid_certificate/test.py::test_connection_accept PASSED test_accept_invalid_certificate/test.py::test_default PASSED test_accept_invalid_certificate/test.py::test_strict_connection_reject PASSED test_accept_invalid_certificate/test.py::test_strict_reject PASSED test_accept_invalid_certificate/test.py::test_strict_reject_with_config PASSED test_drop_is_lock_free/test.py::test_query_is_lock_free[detach part] PASSED test_dictionaries_access/test.py::test_create PASSED test_dictionaries_access/test.py::test_dictget PASSED test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary0-instances_to_check0] PASSED test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Atomic] PASSED test_executable_user_defined_function/test.py::test_executable_function_always_error_python PASSED test_drop_is_lock_free/test.py::test_query_is_lock_free[detach table] PASSED test_dictionaries_access/test.py::test_drop PASSED test_executable_user_defined_function/test.py::test_executable_function_argument_python PASSED test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary1-instances_to_check1] PASSED test_executable_user_defined_function/test.py::test_executable_function_bash PASSED test_drop_is_lock_free/test.py::test_query_is_lock_free[drop part] PASSED test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python PASSED test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Ordinary] PASSED test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash PASSED test_access_control_on_cluster/test.py::test_access_control_on_cluster PASSED test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary2-instances_to_check2] PASSED test_executable_user_defined_function/test.py::test_executable_function_parameter_python PASSED test_access_control_on_cluster/test.py::test_grant_all_on_cluster PASSED test_drop_is_lock_free/test.py::test_query_is_lock_free[truncate] PASSED test_executable_user_defined_function/test.py::test_executable_function_python PASSED test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python PASSED test_access_control_on_cluster/test.py::test_grant_current_database_on_cluster PASSED test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary3-instances_to_check3] PASSED test_executable_user_defined_function/test.py::test_executable_function_signalled_python PASSED test_executable_user_defined_function/test.py::test_executable_function_slow_python PASSED test_executable_user_defined_function/test.py::test_executable_function_sum_json_python PASSED test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3] PASSED test_executable_user_defined_function/test.py::test_executable_function_sum_python PASSED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-NoTx] PASSED test_dictionaries_with_invalid_structure/test.py::test_dictGet_func_for_invalid_dictionary PASSED test_dictionaries_with_invalid_structure/test.py::test_select_from_system_dictionaries_with_invalid_dictionary PASSED test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3_other_bucket] PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_local] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_complex[complex_key_cache] PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3] PASSED test_compression_nested_columns/test.py::test_nested_compression_codec PASSED test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxCommit] PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3_plain] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool PASSED test_groupBitmapAnd_on_distributed/test.py::test_groupBitmapAnd_on_distributed_table PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_local] PASSED test_failed_async_inserts/test.py::test_failed_async_inserts PASSED test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_s3_plain] PASSED test_backup_restore_s3/test.py::test_backup_to_s3 PASSED test_dictionaries_config_reload/test.py::test PASSED test_backup_restore_s3/test.py::test_backup_to_s3_different_credentials PASSED test_consistant_parts_after_move_partition/test.py::test_consistent_part_after_move_partition PASSED test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_simple[cache] PASSED test_backup_restore_s3/test.py::test_backup_to_s3_multipart PASSED test_backup_restore_s3/test.py::test_backup_to_s3_named_collection PASSED test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3] PASSED test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_other_bucket] PASSED test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PART-2020-08-28-20200828_0_0_0] PASSED test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PARTITION-2020-08-27-2020-08-27] PASSED test_extreme_deduplication/test.py::test_deduplication_window_in_seconds PASSED test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_plain_rewritable] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated PASSED test_backup_restore_s3/test.py::test_backup_to_s3_native_copy_multipart PASSED test_backup_restore_s3/test.py::test_backup_to_tar PASSED test_backup_restore_s3/test.py::test_backup_to_tar_bz2 PASSED test_backup_restore_s3/test.py::test_backup_to_tar_gz PASSED test_backup_restore_s3/test.py::test_backup_to_tar_lzma PASSED test_backup_restore_s3/test.py::test_backup_to_tar_xz PASSED test_backup_restore_s3/test.py::test_backup_to_tar_zst PASSED test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic PASSED test_backup_restore_s3/test.py::test_backup_to_zip PASSED test_backup_restore_s3/test.py::test_backup_with_fs_cache[False-True-False] PASSED test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-False-False] PASSED test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-False] PASSED test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-True] PASSED test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability PASSED test_backup_restore_s3/test.py::test_incremental_backup_append_table_def PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas PASSED test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field PASSED test_backup_restore_s3/test.py::test_user_specific_auth PASSED test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool PASSED test_config_reloader_interval/test.py::test_reload_config PASSED test_backup_log/test.py::test_backup_log ================== 15 failed, 85 passed in 1251.39s (0:20:51) ================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 445, in subprocess.check_call(cmd, shell=True) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_i8td71 --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=1e0b53d756cf -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_accept_invalid_certificate/test.py::test_accept test_accept_invalid_certificate/test.py::test_connection_accept test_accept_invalid_certificate/test.py::test_default test_accept_invalid_certificate/test.py::test_strict_connection_reject test_accept_invalid_certificate/test.py::test_strict_reject test_accept_invalid_certificate/test.py::test_strict_reject_with_config test_access_control_on_cluster/test.py::test_access_control_on_cluster test_access_control_on_cluster/test.py::test_grant_all_on_cluster test_access_control_on_cluster/test.py::test_grant_current_database_on_cluster test_async_insert_memory/test.py::test_memory_usage test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field 'test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Atomic]' 'test_attach_partition_with_large_destination/test.py::test_attach_partition_with_large_destination[Ordinary]' test_backup_log/test.py::test_backup_log 'test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3]' 'test_backup_restore_s3/test.py::test_backup_from_s3_to_s3_disk_native_copy[from_s3_to_s3_other_bucket]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_local]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_local_to_s3_plain]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_local]' 'test_backup_restore_s3/test.py::test_backup_to_disk[from_s3_to_s3_plain]' test_backup_restore_s3/test.py::test_backup_to_s3 test_backup_restore_s3/test.py::test_backup_to_s3_different_credentials test_backup_restore_s3/test.py::test_backup_to_s3_multipart test_backup_restore_s3/test.py::test_backup_to_s3_named_collection 'test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3]' 'test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_other_bucket]' 'test_backup_restore_s3/test.py::test_backup_to_s3_native_copy[policy_s3_plain_rewritable]' test_backup_restore_s3/test.py::test_backup_to_s3_native_copy_multipart test_backup_restore_s3/test.py::test_backup_to_tar test_backup_restore_s3/test.py::test_backup_to_tar_bz2 test_backup_restore_s3/test.py::test_backup_to_tar_gz test_backup_restore_s3/test.py::test_backup_to_tar_lzma test_backup_restore_s3/test.py::test_backup_to_tar_xz test_backup_restore_s3/test.py::test_backup_to_tar_zst test_backup_restore_s3/test.py::test_backup_to_zip 'test_backup_restore_s3/test.py::test_backup_with_fs_cache[False-True-False]' 'test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-False-False]' 'test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-False]' 'test_backup_restore_s3/test.py::test_backup_with_fs_cache[True-True-True]' test_backup_restore_s3/test.py::test_incremental_backup_append_table_def test_backup_restore_s3/test.py::test_user_specific_auth test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability test_compression_nested_columns/test.py::test_nested_compression_codec test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool test_config_reloader_interval/test.py::test_reload_config test_consistant_parts_after_move_partition/test.py::test_consistent_part_after_move_partition test_dictionaries_access/test.py::test_create test_dictionaries_access/test.py::test_dictget test_dictionaries_access/test.py::test_drop 'test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_complex[complex_key_cache]' 'test_dictionaries_all_layouts_separate_sources/test_executable_cache.py::test_simple[cache]' test_dictionaries_config_reload/test.py::test 'test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary0-instances_to_check0]' 'test_dictionaries_replace/test.py::test_create_or_replace[atomicdb-instance_to_create_dictionary1-instances_to_check1]' 'test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary2-instances_to_check2]' 'test_dictionaries_replace/test.py::test_create_or_replace[repldb-instance_to_create_dictionary3-instances_to_check3]' test_dictionaries_with_invalid_structure/test.py::test_dictGet_func_for_invalid_dictionary test_dictionaries_with_invalid_structure/test.py::test_select_from_system_dictionaries_with_invalid_dictionary 'test_drop_is_lock_free/test.py::test_query_is_lock_free[detach part]' 'test_drop_is_lock_free/test.py::test_query_is_lock_free[detach table]' 'test_drop_is_lock_free/test.py::test_query_is_lock_free[drop part]' 'test_drop_is_lock_free/test.py::test_query_is_lock_free[truncate]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-NoTx]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxCommit]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxNotFinished]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-part-TxRollback]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-NoTx]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxCommit]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxNotFinished]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[detach-partition-all-TxRollback]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-NoTx]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxCommit]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxNotFinished]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[drop-part-TxRollback]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-NoTx]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxCommit]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxNotFinished]' 'test_drop_is_lock_free/test.py::test_query_is_permanent[truncate-TxRollback]' test_executable_user_defined_function/test.py::test_executable_function_always_error_python test_executable_user_defined_function/test.py::test_executable_function_argument_python test_executable_user_defined_function/test.py::test_executable_function_bash test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash test_executable_user_defined_function/test.py::test_executable_function_parameter_python test_executable_user_defined_function/test.py::test_executable_function_python test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python test_executable_user_defined_function/test.py::test_executable_function_signalled_python test_executable_user_defined_function/test.py::test_executable_function_slow_python test_executable_user_defined_function/test.py::test_executable_function_sum_json_python test_executable_user_defined_function/test.py::test_executable_function_sum_python test_extreme_deduplication/test.py::test_deduplication_window_in_seconds test_failed_async_inserts/test.py::test_failed_async_inserts 'test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PART-2020-08-28-20200828_0_0_0]' 'test_fetch_partition_from_auxiliary_zookeeper/test.py::test_fetch_part_from_allowed_zookeeper[PARTITION-2020-08-27-2020-08-27]' test_groupBitmapAnd_on_distributed/test.py::test_groupBitmapAnd_on_distributed_table -vvv" altinityinfra/integration-tests-runner:37a9815fd2fa ' returned non-zero exit status 1.